Detecting oral temperature using thermal camera

ABSTRACT

A method, system, and computer product for detecting an oral temperature for a human include capturing a thermal image for a human using a camera having a thermal image sensor, detecting a face region of the human from the thermal image, detecting a mouth region on the face region, detecting an open mouth region on the mouth region, detecting a degree of mouth openness on the detected open mouth region, determining that the degree of mouth openness meets a predetermined criterion, and detecting the oral temperature, responsive to determining that the degree of mouth openness meets the predetermined criterion.

FIELD

The present disclosure relates to a method for detecting an oraltemperature for a patient, and more particularly to, a non-invasivemethod, system, and computer product for detecting an oral temperaturefor a patient using a thermal camera.

BACKGROUND

Recently, a home healthcare system has been receiving increasingattention. In such home healthcare system, measurement of a bodytemperature for a patient helps a medical professional (e.g., doctor) tocheck a patient's current health state. In such home healthcare system,a body temperature detection device is generally located at a patient'shome, and thus may require a patient′ frequent manual operation on it.If a patient or a home user of the home body temperature detectiondevice lacks of skill at the operation thereon, there may occur atemperature detection error which may lead the medical professional tomaking a wrong decision on the patient's health state. On the otherhand, many techniques of detecting a body temperature through a faceskin or forehead of a patient have been developed, but it is known thata face skin or forehead temperature is prone to be affected by anambient temperature and an oral temperature provides a relativelyaccurate body temperature.

Thus, there is a need for an oral temperature detection technique thatcan be applied to a home healthcare application where a medicalprofessional is remotely located from a patient.

BRIEF SUMMARY

In an aspect of the present disclosure, a computer-implemented methodfor detecting an oral temperature for a human is provided. The methodincludes capturing a thermal image for a human using a camera having athermal image sensor, detecting a face region of the human from thethermal image, detecting a mouth region on the face region, comparing atemperature value on the mouth region to a reference temperature valueon a first other face region, detecting an open mouth region on themouth region based on a comparison result of the temperature value onthe mouth region to the reference temperature value of the first otherface region, determining whether a mouth of the human is open enough foran oral temperature to be detected, and computing the oral temperaturebased on temperature values on the mouth region and at least one otherface region, responsive to the determination of the mouth being openenough for the oral temperature to be detected.

In an aspect of the present disclosure, a system for detecting an oraltemperature for a human is provided. The system includes a memory devicestoring machine executable program instructions and at least oneprocessing device coupled to the memory device. The at least oneprocessing device is configured to run the machine executable programinstructions to perform capturing a thermal image for a human using acamera having a thermal image sensor, detecting a face region of thehuman from the thermal image, detecting a mouth region on the faceregion, comparing a temperature value on the mouth region to a referencetemperature value on a first other face region, detecting an open mouthregion on the mouth region based on a comparison result of thetemperature value on the mouth region to the reference temperature valueof the first other face region, determining whether a mouth of the humanis open enough for an oral temperature to be detected, and computing theoral temperature based on temperature values on the mouth region and atleast one other face region, responsive to the determination of themouth being open enough for the oral temperature to be detected.

In an aspect of the present disclosure, a computer program productstored in a computer-readable storage medium having computer readableprogram instructions is provided. The computer readable programinstructions are read and carried out by a processing device ofperforming a method for detecting an oral temperature for a human. Themethod includes capturing a thermal image for a human using a camerahaving a thermal image sensor, detecting a face region of the human fromthe thermal image, detecting a mouth region on the face region,comparing a temperature value on the mouth region to a referencetemperature value on a first other face region, detecting an open mouthregion on the mouth region based on a comparison result of thetemperature value on the mouth region to the reference temperature valueof the first other face region, determining whether a mouth of the humanis open enough for an oral temperature to be detected, and computing theoral temperature based on temperature values on the mouth region and atleast one other face region, responsive to the determination of themouth being open enough for the oral temperature to be detected.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a block diagram of an oral temperature detection systemaccording to an exemplary embodiment of the present disclosure;

FIGS. 1B and 1C illustrate examples of images captured in the oraltemperature detection system of FIG. 1A, according to an exemplaryembodiment of the present disclosure;

FIGS. 2A to 2D are flow charts illustrating a method for detecting anoral temperature for an animate object according to an exemplaryembodiment of the present disclosure;

FIG. 3 illustrates a schematic of an example home healthcare networksystem where a face region detection module, an open mouth regiondetection module, a mouth openness detection module, and an oraltemperature detection module are implemented in a communication deviceat a patient's site, according to an exemplary embodiment of the presentdisclosure;

FIG. 4 illustrates a schematic of an example home healthcare networksystem where a face region detection module, an open mouth regiondetection module, a mouth openness detection module, and an oraltemperature detection module are implemented in a communication devicelocated at a medical professional's site, according to an exemplaryembodiment of the present disclosure; and

FIG. 5 is a block diagram of a computing system according to anexemplary embodiment of the present disclosure.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Embodiments of the present invention will now be described in detail onthe basis of the drawings. However, the following embodiments do notrestrict the invention claimed in the claims. Moreover, all combinationsof features described in the embodiments are not necessarily mandatoryfor the architecture of the present invention. Like numbers are assignedto like elements throughout the description of the embodiments of thepresent disclosure.

A method, system, and computer product for detecting an oral temperaturefor an animate object are disclosed. A system for detecting the oraltemperature for the animate object is referred to as an “oraltemperature detection system”. In the present disclosure, an “animateobject” may be understood to include, but is not limited to, a human, apatient, or other warm-blood animals (e.g., mammals) under diagnosis.

FIG. 1A is a block diagram of an oral temperature detection system 1 aaccording to an exemplary embodiment of the present disclosure. FIGS. 1Band 1C illustrate examples of images captured in the oral temperaturedetection system 1 a of FIG. 1A, according to an exemplary embodiment ofthe present disclosure.

Referring to FIG. 1A, an oral temperature detection system 1 a accordingto an exemplary embodiment of the present disclosure may include acamera device 10 for capturing a thermal image 111 for an animate object(not shown), a face region detection module 20 for detecting a faceregion of the animate object from the thermal image 111, an open mouthregion detection module 30 for detecting an open mouth region from thedetected face region, a mouth openness detection module 40 for detectinga mouth openness from the open mouth region, an oral temperaturedetection module 50 for detecting (or calculating) an oral temperatureof the animate object, a display device 60 for displaying the detectedoral temperature, and a controller 70 for controlling operations of thecamera device 10, the face region detection module 20, the open mouthregion detection module 30, the mouth openness detection module 40, theoral temperature detection module 50, and the display device 60.

The camera device 10 may include a thermal image sensing unit 110 forcapturing the thermal image 111 for the animate object. In the contextof the present disclosure, the thermal image 111 may be understood as animage including a plurality of thermal image pixels (e.g., pixels oftemperature values), each of which represents a corresponding one oftemperature values of the image. In one embodiment, the thermal imagesensing unit 110 may be implemented based on an infrared (IR)temperature sensing technique. In some embodiments, the camera device 10may further include an RGB image sensing unit 120 for capturing an RGBimage 121 for the animate object. Thus, the camera device 10 maygenerate the thermal image 111 and/or the RGB image 121. The RGB image121 may represent an actual image of the animate object, which, as anexample, is to be displayed together with a detected oral temperaturevalue (of the animate object) and/or the thermal image 111, to a medicalprocessional (not shown). In one embodiment, the thermal and RGB images111 and 121 have substantially the same number of pixels and the matrixsize as each other.

As shown in FIG. 1A, the face region detection module 20 may receive thethermal image 111 from the camera device 10 and perform a face regiondetection from the thermal image 111. Empirically, the thermal image 111may have thermal distribution related to unique features for a humanface. Examples of the features for a human face may include, but arelimited: 1) a face region may be shown in a thermal image with a highertemperature and denser capillary than other non-facial region such as abackground region; 2) an eye region where temperatures around eyes maybe higher than other regions such as cheeks, a nose bridge, or the like;3) a nose region where a temperature on the nose region is close to abackground temperature (e.g., ambient temperature); and 4) an eye regionhas the highest temperature on the face region. These empiricalinformation allow to derive empirical parameters such as each facefeature′ location, size, thermal indices (e.g., temperature values), orthe like, and such empirical parameters may be stored in memory 80included in the system 1 a and incorporated into a face detection modelwhich facilitates a faster and more accurate face region detection.

To avoid computational complexity and speed up a face region detectionprocess, it is well known that a subset of relevant features (e.g., weakclassifiers) for a subject image must be chosen rather than all possiblefeatures set are chosen. In one embodiment, functions of the face regiondetection module 20 may be implemented using face region detectiontechniques well known in the art based on, but are not limited to, anAdaptive Boosting (Adaboost) algorithm (disclosed in a referenceentitled “Robust real-time face detection”, Paul Viola and MichaelJones, Feb. 20, 2009) and a deep learning algorithm (disclosed in areference entitled “Convolutional face finder: a neural architecture forfast and robust face detection”, Christophe Garcia, IEEE transactions onpattern analysis and machine intelligence, November, 2004). In oneexample, the Adaboost algorithm is used to choose a subset of relevantfeatures and train a series of strong classifiers. The strongclassifiers may be used in a cascaded manner to discriminate non-faceregions from the face region. The detailed algorithms and skills forimplementation of the face region detection module 20 are well known tothose skilled in the art, and are not the main parts of the presentdisclosure, and thus are not described herein. By way of example only,shown in FIG. 1B is an example thermal image 111 and an example image112 captured as a result of face region detection being made on thethermal image 111. Within the image 112 of FIG. 1B, a face region 112 ais shown. The face region detection module 20 may further be configuredto locate face features on the face region 112 a and an example of thelocated face features is shown with a reference number 112 b in FIG. 1B.Shown in FIG. 1B are eye regions 1121 and 1122, a nose region 1123,cheek regions 1124 and 1125, and a mouth region 1126. Shown in FIG. 1Cis an example captured image 113 a for a mouth region.

Referring back to FIG. 1A, the open mouth region detection module 30 maydetect an open mouth region on the mouth region 113 a (FIG. 1C) providedby the face region detection module 20. To this end, the open mouthregion detection module 30 may identify a plurality of thermal imagepixels (not shown) on the mouth region 113 a and compare a respectivetemperature value of each thermal image pixel to a reference temperaturevalue T_ref (see equation 1 below). If one or more image pixels on themouth region 113 a have higher temperature values than the referencetemperature value T_ref, the open mouth region detection module 30 maydetermine a region defined by such one or more image pixels as an openmouth region 113 b, as exemplary shown in FIG. 1C. In one embodiment,the reference temperature value T_ref may be obtained based on atemperature value of other reference face region (e.g., non-mouth faceregion) than the mouth region 113 a. For example, the referencetemperature value T_ref may be a value higher than a temperature valueT_otr of the other reference face region (e.g., cheek region 1124 or1125 of FIG. 1B) by a constant value C, as shown in the followingequation (1):

T_ref=T_otr+C  Equation (1)

In one embodiment, when a cheek region is used as the other referenceface region of which temperature value is compared to a correspondingone of the thermal image pixels within the mouth region 113 a fordetecting an open mouth region 113 b, the constant value C may rangefrom 1.5 to 5.0 degrees Celsius. Stated differently, a region on themouth region 113 a with one or more thermal pixels having highertemperature values by a temperature value C may be determined as an openmouth region 113 b.

Referring still to FIG. 1A, the mouth openness detection module 40 maydetect a mouth openness. Stated differently, the mouth opennessdetection module 40 may determine whether a mouth of an animate objectis open enough such that an oral temperature for the animate object canbe detected. For example, if a mouth of the animate object is openlarger than a predetermined value, the mouth openness detection module40 may determine that the mouth is open enough and an oral temperatureis ready to be detected; otherwise, the mouth openness detection module40 may determine that the mouth is not clearly open enough or the mouthis closed.

To determine the mouth openness, in some embodiments, the mouth opennessdetection module 40 may identify an open mouth region 113 b (FIG. 1C) onthe mouth region 113 a (FIG. 1C), compare a size of the open mouthregion 113 b to a size of the entire face region 113 c (FIG. 1C), anddetermine a ratio of the size of the open mouth region 113 b to the sizeof the entire face region 113 c. When the ratio of the size of theopened mouth region 113 b to the size of the entire face region 113 c isequal to or greater than a reference ratio that ranges, e.g., from 1/17to 1/10, the mouth openness detection module 40 may determine that themouth is open enough for an oral temperature to be detected; otherwisedetermine that the mouth is not clearly open or closed. In comparing thesizes between the open mouth region 113 b and the entire face region 113c, the number of thermal image pixels in each of the open mouth region113 b and the entire face region 113 c may be used. For example, thenumber of thermal image pixels in the open mouth region 113 b may becalculated as correspondingly to the size of the open mouth region 113b, and the number of thermal image pixels in the face region 113 c maybe calculated as correspondingly to the size of the face region 113 c.

By way of example only, if 1/17 is selected as the reference ratio to becompared to the ratio of the size of the opened mouth region 113 b tothe size of the entire face region 113 c, and the open mouth region 113b and the face region 113 c are defined respectively in 10×10(pixel-width×pixel-height) and 40×40 (pixel-width×pixel-height), themouth openness detection module 40 may determine a size of the openmouth region 113 b as 100 pixel region and a size of the face region 113c as 1600 pixel region. In this case, the ratio of the size (i.e., 100pixel region) of the open mouth region 113 b to the size (i.e., 1600pixel region) of the face region 113 c may be 1/16 which is greater than1/17, the reference ratio. Thus, the mouth openness detection module 40may determine that a mouth is open enough for an oral temperature to bedetected and generate a trigger signal 41 that indicates that a mouth isopen.

Referring still to FIG. 1A, the oral temperature detection module 50 maycalculate an oral temperature for an animate object responsive toreceipt of the trigger signal 41, based on the following equation.

T_oral=Tmax_m+α*(Tmax_e−Tmax_n)  Equation (2)

Here, Tmax_m, Tmax_e, and Tmax_n represent maximum temperatures (e.g.,degrees Celsius) on respective mouth region, eye region, and noseregion, and α is a scaling parameter (e.g., 0.5). “*” indicates amultiplication operation. Shown in FIG. 1C is an example image 114 andan oral temperature value 114 a (e.g., 96 degrees Fahrenheit) capturedwhen the oral temperature is detected (or calculated) for the animateobject.

Referring still to FIG. 1A, the display device 60 may display thedetected oral temperature value T_oral together with the thermal image111 (see 114 of FIG. 1C). In some embodiments, although not shown, thedisplay device 60 may display the detected oral temperature value T_oraltogether with the thermal image 111 and the RGB image 121 of the animateobject.

In some embodiments, at least one of the face region detection module20, the open mouth region detection 30, the mouth openness detectionmodule 40, and the oral temperature detection module 50 may beimplemented by a hardware processor (not shown) or based on a(field-programmable gate array) FPGA design (not shown), but in anotherembodiment, implemented based on program codes which are stored inmemory (not shown) or in a hardware processor and executed by thehardware processor.

FIGS. 2A to 2D are flow charts illustrating a method for detecting anoral temperature for an animate object according to an exemplaryembodiment of the present disclosure. As shown in FIG. 2A, the methodaccording to an embodiment of the present disclosure may includecapturing a thermal image (111 of FIG. 1A) and/or an RGB image (e.g.,121 of FIG. 1A) for an animate object using a camera device (10 of FIG.1A) in an operation of step S110. The camera device 10 may capture animage for the animate object.

In addition, as shown in FIG. 2A, in step S120, a face region may bedetected from the thermal image captured through the step S110. In oneembodiment, if the face region is not successfully detected, thecapturing of an image for an animate object (S110) may be repeated (notshown). Next, in step S130, a mouth region may be detected on the faceregion detected through the step S120. As shown in FIG. 2B, the mouthregion may be detected by locating face features on the face region(S131) and detecting the mouth region based on the face features (S132).

Next, referring back to FIG. 2A, a mouth openness may be detected(S140). Stated differently, it is determined whether a mouth is openenough for an oral temperature to be detected. If the mouth isdetermined to be open (S143), an oral temperature may be detected (orcalculated) on the open mouth region (S150); otherwise (S144), the stepsS110, S120, S130, and S140 may be repeated until the mouth is determinedto be open.

As shown in FIG. 2C, the determining of whether a mouth is open enoughfor an oral temperature to be detected (step S140) may further includedetermining an open mouth region on the mouth region (S141). To thisend, as shown in FIG. 2D, a plurality of thermal image pixelscorresponding to the mouth region may be identified (S1411) and arespective temperature value of each thermal image pixel may be comparedto a reference temperature value (S1412). When one or more image pixelson the mouth region have higher temperature values than the referencetemperature value (e.g., T_ref in Equation (1)), a region defined bysuch one or more image pixels may be detected as an open mouth region(S1413); otherwise, such region may be determined as a closed mouthregion (S1414). Next, referring back to FIG. 2C, a size (or area) of theopen mouth region may be compared to a size (or area) of the entire faceregion in step S142. When a mouth is open larger than a predeterminedvalue (YES), it may be determined that the mouth is open enough and anoral temperature is ready to be detected (S143); otherwise (NO), it maybe determined that the mouth is not clearly open or the mouth is closed(S144) and the steps S110, S120, S130, and S140 may be repeated untilthe mouth is determined to be open. In some embodiments, a ratio of thesize of the open mouth region to the size of the entire face region maybe determined. When the ratio of the size of the open mouth region tothe size of the entire face region is equal to or greater than areference ratio that ranges, e.g., from 1/17 to 1/10, it may bedetermined that the mouth is open for an oral temperature to bedetected; otherwise it may be determined that the mouth is not clearlyopen enough or closed. In comparing the sizes between the open mouthregion and the entire face region, the number of thermal image pixels ineach of the open mouth region and the entire face region may be used.For example, the number of thermal image pixels in the open mouth regionmay be calculated as correspondingly to the size of the open mouthregion, and the number of thermal image pixels in the face region may becalculated as correspondingly to the size of the face region.

In the step S150, an oral temperature for an animate object may becalculated using the above-mentioned Equation (2). Next, in step S160,the calculated oral temperature may be displayed using a display device(e.g., 60 of FIG. 1A).

FIG. 3 illustrates a schematic of an example home healthcare networksystem 2 where a face region detection module, an open mouth regiondetection module, a mouth openness detection module, and an oraltemperature detection module are implemented in a communication deviceat a patient's site remote from a medical professional, according to anexemplary embodiment of the present disclosure.

Referring to FIG. 3, a patient 2000 a and a medical professional (e.g.,doctor) 2000 b may be located at a distance and connected through acommunication network 250. For communication over the network 250, eachof the patient 2000 a and the medial professional 2000 b uses at leastone communication device. For illustrative purpose, as shown, acommunication device 200 a may be located at the patient 2000 a's siteand a communication device 200 b may be located at the medicalprofessional 2000 b's site. Stated differently, the communication device200 a may be associated with the patient 2000 a and the communicationdevice 200 b may be associated with the medical professional 2000 bremotely located from the patient 2000 a.

As shown, the communication device 200 a located at the patient 2000 a′site may include a camera device 10 for capturing a thermal image 111for the patient 2000 a, a face region detection module 20 for detectinga face region of the patient 2000 a from the thermal image 111, an openmouth region detection module 30 for detecting an open mouth region onthe detected face region, a mouth openness detection module 40 fordetermining whether the patient 2000 a's mouth is open enough for anoral temperature to be detected, and an oral temperature detectionmodule 50 for detecting (or calculating) an oral temperature for thepatient 2000 a on the open mouth region. Detailed operations orconfigurations of the camera device 10, the face region detection module20, the open mouth region detection module 30, the mouth opennessdetection module 40, and the oral temperature detection module 50 may besubstantially the same as those of FIG. 1A. Duplicate descriptionsthereof will be omitted for the sake of simplicity. The communicationdevice 200 a may further include a network adaptor 90 a forsending/receiving data to/from the communication device 200 b located atthe medical professional 2000 b's site over the communication network250; in some embodiments, the communication device 200 a may beconfigured to send a data signal 2001, which includes an oraltemperature value detected by the oral temperature detection module 50,to the communication device 200 b using the network adaptor 90 a. Thecommunication device 200 a may further include a controller 70 a forcontrolling operations of the camera device 10, the face regiondetection module 20, the open mouth region detection module 30, themouth openness detection module 40, and the oral temperature detectionmodule 50. Thus, the communication device 200 b located at the medicalprofessional 2000 b's may receive the data signal 2001 using a networkadaptor 90 b included therein and display the oral temperature value ofthe patient 2000 a using a display device 60 b included therein. Inanother embodiment, the camera device 10 of the communication device 200a may further capture an RGB image 121 for the patient 2000 a andprovide the RGB image 121 to the network adaptor 90 a, and thus, thedata signal 2001 may further include the RGB image 121 as well as thethermal image 111 and the oral temperature value detected by the oraltemperature detection module 50. Thus, at least one of the RGB image121, the thermal image 111, and the oral temperature for the patient2000 a or any combination(s) thereof may be displayed using the displaydevice 60 b. The communication device 200 b may further include, but isnot limited to, a controller 70 b for controlling the operations of thenetwork adaptor 90 b and the display device 60 b.

Referring to FIG. 3, each of the communication devices 200 a and 200 bmay be implemented with a ultra-mobile personal computer (UMPC), anet-book, a personal digital assistance (PDA), a portable computer (PC),a web tablet, a wireless phone, a mobile phone, a smart phone, ane-book, a portable multimedia player (PMP), a portable game console, anavigation device, a black box, a digital camera, a digital multimediabroadcasting (DMB) player, a digital audio recorder, a digital audioplayer, a digital picture recorder, a digital picture player, a digitalvideo recorder, a digital video player, or the like.

Further, although the camera device 10 is illustrated to be included inthe communication device 200 a, it may be embodied in the communicationdevice 200 a or a peripheral device thereto.

The network 250 may include wired communications based on Internet,local area network (LAN), wide area network (WAN), or the like, orwireless communications based on code division multiple access (CDMA),global system for mobile communication (GSM), wideband CDMA, CDMA-2000,time division multiple access (TDMA), long term evolution (LTE),wireless LAN, Bluetooth, or the like.

FIG. 4 illustrates a schematic of an example home healthcare networksystem 3 where a face region detection module, an open mouth regiondetection module, a mouth openness detection module, and an oraltemperature detection module are implemented in a communication devicelocated at a medical professional site remote from a medicalprofessional, according to an exemplary embodiment of the presentdisclosure.

Referring to FIG. 4, a patient 2000 a and a medical professional 2000 bmay be located at a distance and connected through a communicationnetwork 250. For communication over the network 250, each of the patient2000 a and the medial professional 2000 b uses at least onecommunication device. For illustrative purpose, as shown, acommunication device 300 a may be located at the patient 2000 a's siteand a communication device 300 b may be located at the medicalprofessional 2000 b's site. Stated differently, the communication device300 a may be associated with the patient 2000 a and the communicationdevice 300 b may be associated with the medical professional 2000 bremotely located from the patient 2000 a.

As shown, the communication device 300 a located at the patient 2000 a′site may include, but is not limited to, a camera device 10, a networkadaptor 90 a, and a controller 71 a for controlling operations of thecamera device 10 and the network adaptor 90 a. The camera device 10 maycapture a thermal image 111 for the patient 2000 a and send the capturedthermal image 111 to a network adaptor 90 a. The network adaptor 90 amay be configured to send/receive data to/from the communication device300 b at the medical professional 2000 b's site, over the communicationnetwork 250; in some embodiments, the communication device 300 a may beconfigured to send a data signal 3001, which includes the thermal image111 for the patient 2000 a, to the communication device 300 b using thenetwork adaptor 90 a. Thus, the communication device 300 b located atthe medical professional 2000 b's site may receive the thermal image 111of the data signal 3001 using a network adaptor 90 b included therein.The communication device 300 b may include a face region detectionmodule 20 for detecting a face region of the patient 2000 a from thethermal image 111 received from the communication device 300 a, an openmouth region detection module 30 for detecting an open mouth region onthe detected face region, a mouth openness detection module 40 fordetermining whether the patient 2000 a's mouth is open enough for anoral temperature to be detected, and an oral temperature detectionmodule 50 for detecting (or calculating) an oral temperature for thepatient 2000 a on the open mouth region. Detailed operations orconfigurations of the face region detection module 20, the open mouthregion detection module 30, the mouth openness detection module 40, andthe oral temperature detection module 50 may be substantially the sameas those of FIG. 1A. Duplicate descriptions thereof will be omitted forthe sake of simplicity. Next, the communication device 300 b may displaythe oral temperature value detected by the oral temperature detectionmodule 50 using the display device 60 b included therein. Thecommunication device 300 b may further include a controller 71 b forcontrolling operations of the network adaptor 90 b, the face regiondetection module 20, the open mouth region detection module 30, themouth openness detection module 40, the oral temperature detectionmodule 50, and the display device 60 b. In another embodiment, thecamera device 10 of the communication device 300 a may further capturean RGB image 121 for the patient 2000 a and provide the RGB image 121 tothe network adaptor 90 a, and thus, the data signal 3001 to betransferred to the communication device 300 b may further include theRGB image 121 as well as the thermal image 111. Thus, the communicationdevice 300 b may receive the RGB image 121 and/or the thermal image 111,detect (or calculate) an oral temperature based on the thermal image111, and display at least one of the RGB image 121 and/or the thermalimage 111, and the oral temperature or any combination(s) thereof usingthe display device 60 b.

Referring still to FIG. 4, each of the communication devices 300 a and300 b may be implemented with an UMPC, a net-book, a PDA, a PC, a webtablet, a wireless phone, a mobile phone, a smart phone, an e-book, aPMP, a portable game console, a navigation device, a black box, adigital camera, a DMB player, a digital audio recorder, a digital audioplayer, a digital picture recorder, a digital picture player, a digitalvideo recorder, a digital video player, or the like.

Further, although the camera device 10 is illustrated to be included inthe communication device 300 a, it may be embodied in the communicationdevice 300 a or a peripheral device thereto.

The home healthcare network system 2 or 3 using an oral temperaturedetection system according to an embodiment of the present disclosuredoes not require a patient or home user to manipulate or operate thetemperature detection system, other than to take the thermal and/or RGBimages, which may cause a temperature detection error, insteadfacilitating a real-time oral temperature detection while a medicalprofessional and a patient are being on a conversation (or incommunication), it may minimize the temperature detection error due tothe patient's or home user's immature operating skill on a device andthus allow a medical professional to monitor a patient's health state inreal-time.

FIG. 5 is a block diagram of a computing system 5000 according to anexemplary embodiment of the present disclosure.

Referring to FIG. 5, the computing system 5000 may be used as a platformfor performing (or controlling) the functions or operations describedhereinabove with respect to the systems 1 a, 2, and 3 of FIGS. 1A, 3,and 4 and/or the method of FIGS. 2A to 2D.

In addition, the computing system 5000 may be implemented with an UMPC,a net-book, a PDA, a portable computer (PC), a web tablet, a wirelessphone, a mobile phone, a smart phone, an e-book, a PMP, a portable gameconsole, a navigation device, a black box, a digital camera, a DMBplayer, a digital audio recorder, a digital audio player, a digitalpicture recorder, a digital picture player, a digital video recorder, adigital video player, or the like.

Referring to FIG. 5, the computing system 5000 may include a processor5010, I/O devices 5020, a memory system 5030, a display device 5040, anda network adaptor 5050.

The processor 5010 may drive the I/O devices 5020, the memory system5030, the display device 5040, and the network adaptor 5050 through abus 5060.

The computing system 5000 may include a program module (not shown) forperforming (or controlling) the functions or operations describedhereinabove with respect to the systems 1 a, 2, and 3 of FIGS. 1A, 3,and 4 and/or the method of FIGS. 2A to 2D according to exemplaryembodiments. For example, the program module may include routines,programs, objects, components, logic, data structures, or the like, forperforming particular tasks or implement particular abstract data types.The processor (e.g., 5010) of the computing system 5000 may executeinstructions written in the program module to perform (or control) thefunctions or operations described hereinabove with respect to thesystems 1 a, 2, and 3 of FIGS. 1A, 3, and 4 and/or the method of FIGS.2A to 2D. The program module may be programmed into the integratedcircuits of the processor (e.g., 5010). In an exemplary embodiment, theprogram module may be stored in the memory system (e.g., 5030) or in aremote computer system storage media.

The computing system 5000 may include a variety of computing systemreadable media. Such media may be any available media that is accessibleby the computer system (e.g., 5000), and it may include both volatileand non-volatile media, removable and non-removable media.

The memory system (e.g., 5030) can include computer system readablemedia in the form of volatile memory, such as random access memory (RAM)and/or cache memory or others. The computer system (e.g., 5000) mayfurther include other removable/non-removable, volatile/non-volatilecomputer system storage media.

The computer system (e.g., 5000) can communicate with one or moredevices using the network adapter (e.g., 5050). The network adapter maysupport wired communications based on Internet, LAN, WAN, or the like,or wireless communications based on CDMA, GSM, wideband CDMA, CDMA-2000,TDMA, LTE, wireless LAN, Bluetooth, or the like.

The present invention may be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product may include a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++ or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer readable program instructions may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer readable program instructions by utilizing state information ofthe computer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks may occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the disclosure.As used herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements, if any, in the claims below areintended to include any structure, material, or act for performing thefunction in combination with other claimed elements as specificallyclaimed. The description of the present disclosure has been presentedfor purposes of illustration and description, but is not intended to beexhaustive or limited to the present disclosure in the form disclosed.Many modifications and variations will be apparent to those of ordinaryskill in the art without departing from the scope and spirit of thepresent disclosure. The embodiment was chosen and described in order tobest explain the principles of the present disclosure and the practicalapplication, and to enable others of ordinary skill in the art tounderstand the present disclosure for various embodiments with variousmodifications as are suited to the particular use contemplated.

While the present disclosure has been particularly shown and describedwith respect to preferred embodiments thereof, it will be understood bythose skilled in the art that the foregoing and other changes in formsand details may be made without departing from the spirit and scope ofthe present disclosure. It is therefore intended that the presentdisclosure not be limited to the exact forms and details described andillustrated, but fall within the scope of the appended claims.

1. A computer-implemented method for detecting an oral temperature for ahuman, comprising: capturing a thermal image for a human using a camerahaving a thermal image sensor; detecting a face region of the human fromthe thermal image; detecting a mouth region on the face region;detecting an open mouth region on the mouth region; detecting a degreeof mouth openness on the detected open mouth region; determining thatthe degree of mouth openness meets a predetermined criterion; anddetecting the oral temperature, responsive to determining that thedegree of mouth openness meets the predetermined criterion.
 2. Thecomputer-implemented method of claim 1, wherein the detecting a degreeof mouth openness: comparing a temperature value on the mouth region toa reference temperature value on a first other face region; anddetecting the open mouth region based on a comparison result of thetemperature value on the mouth region to the reference temperature valueof the first other face region.
 3. The computer-implemented method ofclaim 1, wherein the detecting the oral temperature comprises: computingthe oral temperature based on temperature values on the mouth region andat least one other face region.
 4. The computer-implemented method ofclaim 2, wherein the detecting the open mouth region based on acomparison result of the temperature value on the mouth region to thereference temperature value of the first other face region comprises:detecting a first region of the mouth region, a temperature value of thefirst region meeting other predetermined criterion; and determining thefirst region as the open mouth region.
 5. The computer-implementedmethod of claim 4, wherein said other predetermined criterion compriseswhether the temperature value of the first region is higher than thereference temperature value of the first other face region by apredetermined value.
 6. The computer-implemented method of claim 1,wherein the determining that the degree of mouth openness meets apredetermined criterion comprises: determining a first size of the openmouth region and a second size of the face region; determining that thedegree of mouth openness meets said predetermined criterion, responsiveto determining that a ratio of the first size to the second size isequal to or greater than a reference value; determining that the degreeof mouth openness does not meet said predetermined criterion, responsiveto determining that the ratio of the first size to the second size issmaller than the reference value.
 7. The computer-implemented method ofclaim 1, wherein the temperatures values on the open mouth region and onat least one of an eye region and a nose region comprise maximumtemperatures of the respective mouth region, the eye region, and thenose region.
 8. The computer-implemented method of claim 7, wherein thedetecting the oral temperature comprises: calculating the oraltemperature using at least a difference between the maximum temperatureon the eye region and the maximum temperature on the nose region.
 9. Asystem for detecting an oral temperature for a human, comprising: amemory device storing machine executable program instructions; and atleast one processing device coupled to the memory device, the at leastone processing device configured to run the machine executable programinstructions to perform: capturing a thermal image for a human using acamera having a thermal image sensor; detecting a face region of thehuman from the thermal image; detecting a mouth region on the faceregion; detecting an open mouth region on the mouth region; detecting adegree of mouth openness on the detected open mouth region; determiningthat the degree of mouth openness meets a predetermined criterion; anddetecting the oral temperature, responsive to determining that thedegree of mouth openness meets the predetermined criterion.
 10. Thesystem of claim 9, wherein to detect a degree of mouth openness, the atleast one processing device is further configured to run the machineexecutable program instructions to perform: comparing a temperaturevalue on the mouth region to a reference temperature value on a firstother face region; and detecting the open mouth region based on acomparison result of the temperature value on the mouth region to thereference temperature value of the first other face region.
 11. Thesystem of claim 9, wherein to determine the oral temperature, the atleast one processing device is further configured to run the machineexecutable program instructions to perform: computing the oraltemperature based on temperature values on the mouth region and at leastone other face region.
 12. The system of claim 10, wherein to detect theopen mouth region based on a comparison result of the temperature valueon the mouth region to the reference temperature value of the firstother face region, the at least one processing device is furtherconfigured to run the machine executable program instructions toperform: detecting a first region of the mouth region, a temperaturevalue of the first region meeting other predetermined criterion; anddetermining the first region as the open mouth region.
 13. The system ofclaim 12, wherein said other predetermined criterion comprises whetherthe temperature value of the first region is higher than the referencetemperature value of the first other face region by a predeterminedvalue.
 14. The system of claim 9, wherein to determine that the degreeof mouth openness meets said predetermined criterion, the at least oneprocessing device is further configured to run the machine executableprogram instructions to perform: determining a first size of the openmouth region and a second size of the face region; determining that thedegree of mouth openness meets said predetermined criterion, responsiveto determining that a ratio of the first size to the second size isequal to or greater than a reference value; determining that the degreeof mouth openness does not meet said predetermined criterion, responsiveto determining that the ratio of the first size to the second size issmaller than the reference value.
 15. The system of claim 9, wherein thetemperatures values on the open mouth region and on at least one of aneye region and a nose region comprise maximum temperatures of therespective mouth region, the eye region, and the nose region.
 16. Acomputer program product stored in a non-transitory computer-readablestorage medium having computer readable program instructions, thecomputer readable program instructions read and carried out by aprocessing device of performing a method for detecting an oraltemperature for a human, wherein the method comprises: capturing athermal image for a human using a camera having a thermal image sensor;detecting a face region of the human from the thermal image; detecting amouth region on the face region; detecting an open mouth region on themouth region; detecting a degree of mouth openness on the detected openmouth region; determining that the degree of mouth openness meets apredetermined criterion; and detecting the oral temperature, responsiveto determining that the degree of mouth openness meets the predeterminedcriterion.
 17. The computer product of claim 16, wherein the detecting adegree of mouth openness: comparing a temperature value on the mouthregion to a reference temperature value on a first other face region;and detecting the open mouth region based on a comparison result of thetemperature value on the mouth region to the reference temperature valueof the first other face region.
 18. The computer product of claim 16,wherein the detecting the oral temperature comprises: computing the oraltemperature based on temperature values on the mouth region and at leastone other face region.
 19. The computer product of claim 17, wherein thedetecting the open mouth region based on a comparison result of thetemperature value on the mouth region to the reference temperature valueof the first other face region comprises: detecting a first region ofthe mouth region, a temperature value of the first region meeting otherpredetermined criterion; and determining the first region as the openmouth region.
 20. The computer product of claim 19, wherein said otherpredetermined criterion comprises whether the temperature value of thefirst region is higher than the reference temperature value of the firstother face region by a predetermined value.